Updating github actions and requirements #257

jellis18 · 2021-03-13T23:10:38Z

This PR does the following:

Revamps github actions tests to run on both linux and osx for python 3.6 - 3.8
Adds a build and release target to github actions. The build will build the wheels and release pushes them to PYPI. I don't know if there is a NANOGrav pypi account but that should probably be used. These build and release actions get triggered on a github release
Added isort pre-commit. This will ensure that imports are always consistent
Replaced print with loggers where appropriate
Removed all __future__ since python 2.7 support is dropped.

There are still a few things that will need to be taken care of down the line. First off, we are still installing libstempo from github and not from a pypi release. I've been working with @vallis on getting a pypi release ready so that will be along soon.

Next, scikit-sparse still requires numpy and cython to be installed beforehand because they don't have the right build dependencies set. I made a PR to fix this but it may not get any attention (the repo hasn't been touched for a few years). Another option is to maybe fork it and use that for our scikit-sparse install (I already did this but it may be better to try to work through the scikit-sparse devs first).

Lastly, all of these changes are mainly meant for the pip path. Once there is a version on pypi we can then make a conda-forge version.

* Updating github actions * Update CI * add numpy cython install * Check brew list * Automake * gfortran * alias * environment variable * brew link * try again * one block * fix typo * indent * remove numpy and cython from pyproject * change jellis-> vallis and add python requirement * Using logger and adding isort * isort config * logger.info -> print in docsting * Adding wheels and pypi * Adding "build requirements" for scikit-sparse

codecov · 2021-03-13T23:37:34Z

Codecov Report

Merging #257 (ce403e7) into master (bec52e0) will increase coverage by 0.30%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #257      +/-   ##
==========================================
+ Coverage   85.66%   85.97%   +0.30%     
==========================================
  Files          12       12              
  Lines        2748     2737      -11     
==========================================
- Hits         2354     2353       -1     
+ Misses        394      384      -10

Impacted Files	Coverage Δ
enterprise/signals/selections.py	`87.14% <ø> (-0.19%)`	⬇️
enterprise/signals/white_signals.py	`98.41% <ø> (-0.02%)`	⬇️
enterprise/pulsar.py	`91.46% <100.00%> (+0.77%)`	⬆️
enterprise/signals/signal_base.py	`87.65% <100.00%> (+0.56%)`	⬆️
enterprise/signals/utils.py	`84.77% <100.00%> (+0.43%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bec52e0...ce403e7. Read the comment docs.

setup.py

vallis · 2021-03-23T22:55:24Z

This looks great Justin. Thank you so much. Several things:

The libstempo release (2.4.0) is now up, if you want to switch the install to that.
We had problems with setting up the coverage requirements, so I had disabled it as a check. If you know how to fix it with reasonable criteria, please do.
In addition to numpy/cython, the problem with scikit-sparse seems to be suitesparse. We need users to have a decent/fast version of that, otherwise there's no purpose in using scikit-sparse to begin with. Perhaps we can discuss this in scikit-sparse install problems don't fail meaningfully #239.

jellis18 · 2021-03-24T00:58:05Z

@vallis if you just want coverage percent as a check I added that in this PR directly as part of the tests.

As for suitesparse I was unaware there were different builds of the main library itself. I thought it just had to do with whether or not a mkl numpy version was used to build scikit-sparse.

Unfortunately I seems nearly impossible to get mkl-based numpy though pip. I think the best approach may be to leave as is for the pip install path and just use conda if you care about speed.

Doesn't look like conda-forge does anything special for suitesparse

How have you guys handled this in the past?

jellis18 · 2021-03-25T12:21:42Z

enterprise/pulsar.py

 try:
    import libstempo as t2
 except ImportError:
-    print("Ooh, no libstempo?")
+    logger.warning("libstempo not installed. Will use PINT instead.")  # pragma: no cover


@nanograv/enterprise These are not really needed if we have both libstempo and PINT as dependencies. Do we want to keep them both or have one as optional?

I'm glad you changed them to logger warnings. I think we could make the case either way. People only need one of the packages to run enterprise. I think the only dependency on either package right now is within enterprise.Pulsar, so users can get a way with only using one. I guess that is a pretty rare situation for software. Maybe we just raise an ImportError if neither are present?

Well I think there are 3 options:

Don't require either as base dependencies and have a pip install enterprise-pulsar[libstempo] and pip install enterprise-pulsar[pint] path. I don't really like this because it could lead to broken installs because the package won't really work without one or the other.

Choose one as the default and add the other as an extra. Kind of like this one but not sure which should be default

Leave both as it is now. This is fine too

Actually the whole Pulsar function/class is a pretty bad design. I don't think I knew much about class polymorphism at the time. Its too much to refactor it for this PR but it could definitely be better.

jellis18 · 2021-03-25T13:53:21Z

@nanograv/enterprise I don't know how to turn off these checks above that aren't relevant...

Hazboun6 · 2021-03-25T22:00:02Z

@nanograv/enterprise I don't know how to turn off these checks above that aren't relevant...

I think we can delete them from .github/.codecov.yaml. I assume we could just delete

patch:
      default:
        target: auto
        threshold: 6%

Then there will be no patch coverage check. You are rerunning tests now, so I think those were the tests that weren't passing, is that correct?

jellis18 · 2021-03-25T22:03:27Z

@Hazboun6 the tests all pass now but yes we could remove that patch coverage check. What I don't know how to get rid of are these

They don't run on pull requests and I don't even know where "lint" comes from as its not a target

Actually I think I see now. Those were the old targets but aren't there anymore

Hazboun6 · 2021-03-25T22:08:32Z

Actually I think I see now. Those were the old targets but aren't there anymore

Yeah, @jellis18 is this because GitHub Actions runs the old tests and the new tests in a PR?

Hazboun6

@jellis18 This all looks great. Thanks for doing the hard work of getting the requirements set up correctly.

Hazboun6 · 2021-03-26T15:59:33Z

@jellis18 This is super weird. The Action workflow page doesn't even show those checks. Really annoying that we can't circumvent them somehow.

jellis18 · 2021-03-26T22:01:22Z

@Hazboun6, fixed it

jellis18 added 21 commits March 7, 2021 11:44

Updating github actions

89d3fe7

Update CI

2135633

add numpy cython install

645e350

Check brew list

b52dada

Automake

264ac0f

gfortran

786b6bd

alias

01ad509

environment variable

2d5d411

brew link

819625e

try again

1fe00fa

one block

cbc877d

fix typo

c5a2962

indent

c0c639b

remove numpy and cython from pyproject

0ef8c1e

change jellis-> vallis and add python requirement

c76dcf1

Using logger and adding isort

76a5f7f

isort config

72fbfe4

logger.info -> print in docsting

6f8926c

Adding wheels and pypi

2b36b1e

Adding "build requirements" for scikit-sparse

7bf6385

jellis18 requested review from Hazboun6, vallis and paulthebaker March 13, 2021 23:10

jellis18 added 4 commits March 14, 2021 09:23

Naming to enterprise-pulsar for pypi

d930cbf

Merge branch 'master' of https://github.com/jellis18/enterprise

e9a2aa5

Small changes to makefile

104594f

Add coverage percent to test

9e80721

jellis18 commented Mar 21, 2021

View reviewed changes

setup.py Outdated Show resolved Hide resolved

More cleaning of makefile

e0e7672

This was referenced Mar 21, 2021

Remove soft replacement for missing sksparse #258

Closed

put reqs in setup.py #198

Closed

scikit-sparse install problems don't fail meaningfully #239

Open

jellis18 added 3 commits March 21, 2021 09:01

log-level info in tests

171d56c

Udating contributing

3b3c8c1

Fix contributing.rst

4b61097

Adding libstempo pypi and dropping CI fail for coverage

2f0021a

Hazboun6 mentioned this pull request Mar 24, 2021

Appending Pint's toas and model classes to PintPulsar #259

Merged

jellis18 added 2 commits March 25, 2021 08:04

adding no-cover to exceptions

f35c2a3

Updated test

d46f2e5

jellis18 commented Mar 25, 2021

View reviewed changes

Merge branch 'master' of https://github.com/nanograv/enterprise

ce403e7

Hazboun6 approved these changes Mar 25, 2021

View reviewed changes

vallis merged commit 90e976c into nanograv:master Mar 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating github actions and requirements #257

Updating github actions and requirements #257

jellis18 commented Mar 13, 2021 •

edited

Loading

codecov bot commented Mar 13, 2021 •

edited

Loading

vallis commented Mar 23, 2021

jellis18 commented Mar 24, 2021

jellis18 Mar 25, 2021

Hazboun6 Mar 25, 2021

jellis18 Mar 26, 2021

jellis18 Mar 26, 2021

jellis18 commented Mar 25, 2021

Hazboun6 commented Mar 25, 2021

jellis18 commented Mar 25, 2021 •

edited

Loading

Hazboun6 commented Mar 25, 2021 •

edited

Loading

Hazboun6 left a comment

Hazboun6 commented Mar 26, 2021

jellis18 commented Mar 26, 2021

Updating github actions and requirements #257

Updating github actions and requirements #257

Conversation

jellis18 commented Mar 13, 2021 • edited Loading

codecov bot commented Mar 13, 2021 • edited Loading

Codecov Report

vallis commented Mar 23, 2021

jellis18 commented Mar 24, 2021

jellis18 Mar 25, 2021

Choose a reason for hiding this comment

Hazboun6 Mar 25, 2021

Choose a reason for hiding this comment

jellis18 Mar 26, 2021

Choose a reason for hiding this comment

jellis18 Mar 26, 2021

Choose a reason for hiding this comment

jellis18 commented Mar 25, 2021

Hazboun6 commented Mar 25, 2021

jellis18 commented Mar 25, 2021 • edited Loading

Hazboun6 commented Mar 25, 2021 • edited Loading

Hazboun6 left a comment

Choose a reason for hiding this comment

Hazboun6 commented Mar 26, 2021

jellis18 commented Mar 26, 2021

jellis18 commented Mar 13, 2021 •

edited

Loading

codecov bot commented Mar 13, 2021 •

edited

Loading

jellis18 commented Mar 25, 2021 •

edited

Loading

Hazboun6 commented Mar 25, 2021 •

edited

Loading